Practical Implications of Using Different Tests of Measurement Invariance for Polytomous Measures

Using male/female and Caucasian/African American comparison groups, this study examined the practical ramifications of using two IRT-based analytic methods, DFIT and the Likelihood Ratio Test (LRT), in assessing the measurement invariance of a 21-item leadership development scale under ten sample size conditions (e.g., 200, 500, & 1000). In nine of ten conditions, the LRT identified multiple items that exhibited DIF whereas DFIT only detected a single item with DIF in one set of analyses. Conclusions based on the LRT indicated a lack of measurement invariance for the scale, while DFIT implied near perfect measurement invariance. Thus, these findings highlight the implications of choice of analytic method on the determination of measurement invariance in applied samples.